<html>
<head><meta charset="utf-8"><title>Attribute stripping for proc macros · t-compiler/rust-analyzer · Zulip Chat Archive</title></head>
<h2>Stream: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/index.html">t-compiler/rust-analyzer</a></h2>
<h3>Topic: <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html">Attribute stripping for proc macros</a></h3>

<hr>

<base href="https://rust-lang.zulipchat.com">

<head><link href="https://rust-lang.github.io/zulip_archive/style.css" rel="stylesheet"></head>

<a name="237179615"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237179615" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237179615">(May 03 2021 at 15:24)</a>:</h4>
<p>In <a href="https://github.com/rust-analyzer/rust-analyzer/issues/8434">https://github.com/rust-analyzer/rust-analyzer/issues/8434</a>, I describe that we have to remove some attributes from custom derive input before calling the proc macro with it. The same applies to attribute macros, for example:</p>
<div class="codehilite" data-code-language="Rust"><pre><span></span><code><span class="cp">#[inline]</span><span class="w"></span>
<span class="cp">#[proc_macro]</span><span class="w"></span>
<span class="cp">#[cold]</span><span class="w"></span>
<span class="k">fn</span> <span class="nf">f</span><span class="p">()</span><span class="w"> </span><span class="p">{}</span><span class="w"></span>
</code></pre></div>
<p>Then <code>proc_macro</code>'s input consists of <code>#[inline] #[cold] fn f() {}</code>, and the attribute macro was removed. This is important because otherwise many proc macros will result in infinite expansion if it's left in place.</p>
<p>I tried to implement this in <a href="https://github.com/rust-analyzer/rust-analyzer/pull/8443">https://github.com/rust-analyzer/rust-analyzer/pull/8443</a> but later realized that <code>hir_def</code> needs a better way to tell <code>hir_expand</code> which attributes to remove. Specifically, <code>hir_def</code>'s <code>Attr</code> abstraction turns doc comments into attributes, but <code>hir_expand</code> doesn't know that. The solution I'd propose here is:</p>
<ul>
<li>move parts of <code>attr.rs</code> from <code>hir_def</code> to <code>hir_expand</code></li>
<li>move <code>intern.rs</code> to <code>hir_expand</code>, since it's required by <code>attr.rs</code></li>
<li>use the same abstraction to derive attribute indices and to obtain and remove the corresponding syntax</li>
</ul>
<p>Another alternative would be to do attribute stripping entirely in <code>hir_def</code>, but I feel like that shouldn't be its responsibility.</p>
<p>Any opinions on this?</p>



<a name="237180270"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237180270" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Edwin Cheng <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237180270">(May 03 2021 at 15:29)</a>:</h4>
<p>IIRC, seem like move parts of <code>attr.rs</code> should be simpler option here</p>



<a name="237189886"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237189886" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237189886">(May 03 2021 at 16:36)</a>:</h4>
<p>Hm, why does <code>hir_def</code> handling of doc comments affect macros?</p>
<p>I think these might be two different things? Macros want to covert <code>///</code> to <code>#[]</code> token trees during lowering. Hir wants to convert <code>///</code> to attributues during analysis. </p>
<p>I don't think <code>hir</code> necessary needs  to go via <code>#[]</code>, it can convert commets directly to hir::Attrs</p>



<a name="237190368"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237190368" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237190368">(May 03 2021 at 16:40)</a>:</h4>
<p>It makes it hard to find the right AST node to remove, because now "attribute index 4" also counts doc comments</p>



<a name="237190418"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237190418" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237190418">(May 03 2021 at 16:41)</a>:</h4>
<p>If we want to replicate that logic in <code>hir_expand</code>, copying the relevant parts of <code>attr.rs</code> there seems like the right solution</p>



<a name="237194452"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237194452" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237194452">(May 03 2021 at 17:10)</a>:</h4>
<p>Hm, can we pass the right index to <code>hir_expand</code>?</p>



<a name="237194501"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237194501" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237194501">(May 03 2021 at 17:10)</a>:</h4>
<p>or rather, can we avoid indicis alltogether, and pass whole SyntaxNodes?</p>



<a name="237194565"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237194565" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237194565">(May 03 2021 at 17:11)</a>:</h4>
<p>So you'd have something like <code>to_tt(node: ast::Struct, attrs_to_skip: HashSet&lt;ast::Attr&gt;) -&gt; tt::TokenTree</code> ?</p>



<a name="237196513"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237196513" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237196513">(May 03 2021 at 17:25)</a>:</h4>
<p>that would work, but <code>hir_expand</code> computes the macro input, so we'd have to restructure how the queries work</p>



<a name="237197677"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237197677" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237197677">(May 03 2021 at 17:32)</a>:</h4>
<p>ouch, right. So we'd need to embedd that into <code>MacroCallId</code>, and that needs ids...</p>



<a name="237198323"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237198323" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237198323">(May 03 2021 at 17:36)</a>:</h4>
<p>I guess, my gut feeling is that it makes sense to keep Attrs in the <code>def</code>, and adjust expand's interface such that it is natural for the expand (so, use ids to refer to syntactic attributes, rather than <code>hir_def::Attr</code> attributes), but I am also not that much familiar with the code here.</p>



<a name="237199087"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237199087" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237199087">(May 03 2021 at 17:40)</a>:</h4>
<p>I suppose we could do that, yeah. Perhaps by making <code>AttrId</code> store whether it's a doc comment or an attribute or something...</p>



<a name="237201117"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237201117" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237201117">(May 03 2021 at 17:53)</a>:</h4>
<p>I guess we can avoid <code>AttrId</code>s altogether for this purpose? Like, can we move <code>AttrId</code> to <code>hir_def</code>, and use just <code>ast_attr_position: usize</code> in expand?</p>



<a name="237201136"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/237201136" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#237201136">(May 03 2021 at 17:53)</a>:</h4>
<p>We don't need to use the same type for two</p>



<a name="238144654"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/238144654" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#238144654">(May 10 2021 at 13:48)</a>:</h4>
<p>One thing I'm not sure about here is how to derive the AST attribute index from <code>AttrId</code> in the first place (since we are already dealing with semantic <code>Attrs</code> when we need to know the attribute to remove). Doing that also goes back-and-forth between the representations a bit much for my taste.</p>



<a name="238149433"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/238149433" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#238149433">(May 10 2021 at 14:18)</a>:</h4>
<p>I think the easiest way to do this is to encode <code>AttrId</code> as <code>(kind: DocCommentOrRealAttr, syntactic_index: u32)</code>, that makes it trivial</p>



<a name="238149516"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/238149516" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#238149516">(May 10 2021 at 14:19)</a>:</h4>
<p>Only it probably doesn't work right with out-of-line modules, but I'm pretty sure hir_def normally doesn't respect inner attributes in any case</p>



<a name="238151438"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/238151438" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#238151438">(May 10 2021 at 14:30)</a>:</h4>
<p><span aria-label="thinking" class="emoji emoji-1f914" role="img" title="thinking">:thinking:</span> my general rule of thumb is that back&amp;forth is generally worth it to keep layering sane (rust-analyzer has a lot of that, lsp_types -&gt; ide::* -&gt; hir::* -&gt; hir_def::*, I think it pays off).</p>
<p>On the other hand, its not like we worry about layering a lot at <em>this</em> layer , so if you feel like that's too much layering, feel free to poke a hole in lasagna !</p>



<a name="238151921"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/238151921" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#238151921">(May 10 2021 at 14:33)</a>:</h4>
<p>that layering has benefits though, I don't really see them for <code>AttrId</code> conversion</p>



<a name="239651966"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/239651966" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#239651966">(May 20 2021 at 20:10)</a>:</h4>
<p>this turned out not to work so well because inner attributes require special handling that's done in <code>attr.rs</code></p>



<a name="239681273"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/239681273" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> Jonas Schievink  [he/him] <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#239681273">(May 21 2021 at 00:52)</a>:</h4>
<p>should we maybe move a version of <code>hir_def::attr::collect_attrs</code> into libsyntax? seems like an adequate place to answer "which syntactical attributes are on this item?"</p>



<a name="239710597"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/239710597" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#239710597">(May 21 2021 at 07:54)</a>:</h4>
<p>uhu</p>



<a name="239710629"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/239710629" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#239710629">(May 21 2021 at 07:54)</a>:</h4>
<p>I think <code>AttrsOwner</code> API should be extended, so it has methonds like</p>
<div class="codehilite"><pre><span></span><code>
</code></pre></div>



<a name="239710739"></a>
<h4><a href="https://rust-lang.zulipchat.com#narrow/stream/185405-t-compiler/rust-analyzer/topic/Attribute%20stripping%20for%20proc%20macros/near/239710739" class="zl"><img src="https://rust-lang.github.io/zulip_archive/assets/img/zulip.svg" alt="view this post on Zulip" style="width:20px;height:20px;"></a> matklad <a href="https://rust-lang.github.io/zulip_archive/stream/185405-t-compiler/rust-analyzer/topic/Attribute.20stripping.20for.20proc.20macros.html#239710739">(May 21 2021 at 07:55)</a>:</h4>
<div class="codehilite"><pre><span></span><code>fn inner_attrs_and_doc_comments()
fn outer_attrs_and_doc_comments()
fn attrs_and_doc_coments()
</code></pre></div>



<hr><p>Last updated: Aug 07 2021 at 22:04 UTC</p>
</html>